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A Computerized Method of Longitudinal 
Evaluation of Student Performance* 

By Jon Gosser** 

The most important question to every teacher is, 'Am 
I doing a good job teaching?” 1 In order to partially answer 
the above question the individual teacher needs to define 
teaching (independent variable), the effects of good teach- 
ing (dependent variable), and the method to measure the 

effectiveness of his tea.ching. 

The independent variable (teaching) can be best defined 
as a urocess which consists of the teachers behavior, tne 
students* behavior, the physical setting, and the behavior o± 
the supportive personnel. Thus the teachers’ benavior is only 
one part of the teaching process. (See Figure 1) However, 
the teacher is the formal leader and therefore considered 
responsible for the entire teaching process of which he is a 
part. In most cases, the individual teacher cannot control to 
any significant degree the other parts of the teaching process. 

* Without the assistance of Mr. Ben Paulson and Delta College’s 
Computer facilities this paper would not have been possible. 
Contributions and support of Dr- Ear bans Lai and Delta’s 
Learning Resources Center are also greatfully acknowledged. 
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Instructor of Psychology, Delta College, University Center, 
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2 Gosser 

as defined above. Thus our independent variable is an ill- 
defined and ambiguous one. The obvious "constant' 1 in this 
independent variable is the teacher himself. 

Due to the ambiguous and ill-defined nature of our 
independent variable the approach described here will not 
attempt to ascertain causal relationships as normally defined 
in science. What we can do, however, is to precisely measure 
and describe the effects of the total process (independent 
variable) without being able to e-entribute these effects to 
any % --part i cular part of the teaching process. In most cases 
the teacher, as formal leader of the teaching process, will : 
probably be assumed to produce the desirable or undesirable 

effects. 

The dependent variable (the effects of good teaching) has 
been looked upon .in such terms as its effects upon the teacher, 
the school administration, political beliefs, religious values, 
parents, the amount of tax and other monetary support for 
education, national survival, and various types of student 
behavior. All of the above criteria may be valla and justifi- 
able outcomes of the teaching process, but student behaviors 
are felt by most individual teachers to be the most important. 

We can break most student behavior into four broad types: 

(1) student responses to questions and statements about the course 
content (achievement), (2) student responses to questions and 
statements about the teaching process, recording conditions, 
and parts thereof (student opinion of teaching), (3) student 
responses to his own and other student responses in 1 and 2 
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above, and. (4) criterion situations outside the educational 
setting (job performance). (See Figure 2) 

Our dependent variable (student behavior) can only be 
precisely defined by including thousands, if not millions, of 
specific small responses to specific situations. For example, 
in student resoonses to course content in an elementary math 



class the response to the question 2 + 2 = is a different 
response than +2 , as the student may respond correctly to 
the one format but incorrectly to the other format. Thus the 
only conceivable way at present to deal with these thousands, 
if not millions, of different student responses is through the 
use of .high speed digital computers and appropriate memory 
devices. Since as teachers we clearly are not primarily 
interested in the responses of our students in a given semester, 
but are primarily interested in how our students wail respond 



ten years later and also, we are not primarily interested in 
how well we have taught in a given semester- but whether or not 



we are teaching more effectively now than we were last year or 
ten years before. This interest in long term effects and 
comparisons requires the use of the longitudinal approach. 

In. essence, what we have said thus far is that teachers, 
as leaders of the teaching process (independent variable), want 
to know what effects they and the teaching process are having 
on student responses to academic content, opinions of teachers, 
student opinion of other students, and long term retention 
and transfer of these behaviors. Due to the number of dilferent 






student responses, of interest to teachers, and the longitu 
dinal interest in their students* behavior, the use of compu- 
terized and longitudinal approach to the problem is necessary. 

METHOD 

Since student responses to questions and statements about 
the teaching process and components thereof are relatively few 
in number and easy to obtain, we chose student responses to 53 
questions about the teaching process. These questions were 
taken from, "The University of Washington Survey of Student 
Opinion of Teaching," 2 and from, "The Purdue Rating Scale for 
Instruction."- 3 In addition, we added some questions and state- 
ments of particular relevance to us. Since we used different 
instructions for the two questionnaires (see Figure 3 and 4), 
the students responses to these questions were coded in the 
following ways: (1) a number was assigned to each completed 

questionnaire for verification of the accuracy of the key 
punching, (2) the class in which the student filled out the 
questionnaire, (-3) the time of the semester in which he res 
ponded to the questionnaire (mid-term, final), (4) the actual 
date on which he responded, (5) the number of the question to 
which he responded, and (6) name of the questionnaire. (See 

Figure 5) 

' TTo attempt will be made here to go into the details of how 
the questionnaires were administered. Neither will we go into 
the systems and programs necessary to carry out a computerized 
longitudinal description and analysis of student responses to 
questions and statements about the teaching process „ 
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RESULTS 

The result# to date is the ability to describe anj analyze 
student responses to questions and statements about the teach- 
ing process. The basic usefulness of these descriptions is 
that they document, in a reliable and valid fashion, one of the 
effects of the teaching process. In otherwords, we have gener- 
ated an objective description of one of the products we as 
teachers produce. The remainder of the result section will 
describe and illustrate some nonrandom samples of descriptions 
and a-nalyses. 

In Figure 6, we see that the median response to the question, 
"Do you think that your instructor should have asked your opinion 
of this' course, ” was yes, at the end of the semester in the course 
evaluation when combining all classes in analyzing the results 
by semester. Figure 7 shows, that the median (MD.) response to 
the above question was yes, for each of the 31 individual sections 
which were grouped by semester in Figure 6. 

In Figure 8, we see that .the median response to the 

question, "Personal appearance," at the end of the semester in 

* 

the instructor evaluation was superior for all classes grouped 
by semester. In Figure 9 » we see that looking at the results by 
class for the "Personal appearance" question that only 26 out 
of the 31 classes had a median response of superior. Two of the 
*5 remaining classes had a median rating of competent and the 
other 3, a median response of outstanding. This shows that the 

value of being able to look not only at the results summarized 

# 

by semester but also by section or class. 
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In Figure 10, .we see that the median response at the end 
of the semester, of general psychology classes only, grouped 
by semester, to the statement, "Amount of freedom allowed 
students in the section of the materials to be studied (con- 
sidering the subject matter),” was above average for the first 
3 semesters. While during the last five semesters the median 
was either average or below average. This decrease in amount 
of freedom the students perceived was in agreement with the 
changes that were made in teaching methods at the start of the 

fourth semester. 

These sample results indicate the type of information 
presently available on all 53 questions. Tests of significance 
and other inferential statistical analyses have not at present 
been integrated into the system. But there would oe no necess 
ity for any changes in the data itself to run any statistical 
tests desired automatically by the computer. 



DISCUSSION 

We have described student responses to questions and 

statements about the teaching process grouped by classes, 

semesters, and subjects. The descriptions could have been 

grouped in many different ways dependent upon the interest of 

the individual instructor. Actually, we may organize the data 

in terms of any of the responses which students can/or did make. 

For example, the opinions of the males versus all the females, 

the opinions of all of those who expect to receive an ”A” , 

and those who learn things of no practical value and had no 

* 

opinion on taking another course under the instructor and who 
were female. Another example might be, how did the students 
who responded without stating an opinion to the .quest ions 
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on, “ individual goals," respond to, "I am learning things of 
practical value, some practical value, or no practical value. 

This system of computerized longitudinal evaluation of student 
performance can he applied to any and all identifiaole and 

recordable student responses s 

The time .and money involved in computerized systems can 
be quite large but with appropriate systems and programming can 
be considerably reduced. For example, a way to have done the 
analyses described in the result section would have been to 
look at each of 3,332 records until you find the first record you 
want and then to look through all 3»332 records again to find the 
next rec-ord you wanted, etc. This would have required roughly 60 
hours of IBM 360-40 computer time or nearly $4,500 for one report. 
The way this report was generated required less than 2 hours or 
only 4 ? 1 3 0 worth of computer time. With sufficient changes in 
the programs the cost can probably be reduced to $50 or less; 

SUMMARY 

This paper has tried to indicate some of the ways we as 
teachers can obtain partial answers to the question, "Am I doing 
a good job in teaching?" In essence, we have said we can pre- 
sently describe in great detail the effects of the teaching pro- 
cess on student responses. That is, we can keep an inventory of 
the responses our students make to (1) questions and statements 
about subject matter, (2) questions and statements on the teaching 
process, and (3) questions and statements about theirs 1 and other 
students* responses, and (4) criterion situations outside of tne 



educational setting. 
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STUDENT PERFORMANCES 



Student Responses To: 




1. Questions and statements about course content 
(achievement) 

Ex. 1. In an experiment designed to study the 
effect of background music on production in 
a factory, the background music is the: 

1 . dependent variable 
Z. intervening variable 
3* only relevant variable 
h» independent variable. 




U* 




Questions and statements about the teaching 
process, recording conditions, and parts 
thereof. 

Ex. 1. The degree to which the objectives of 
the course were clarified and discussed. 

1. excellent 1;. Below average 

2. above average extremely poor. 

3. average 

Criterion situations outside of the 
educational setting. 

Ex. ”How many articles have you published?” 

His own and other student’s responses made 
above. 

Ex. How sure are you that your answer 
is correct? 

1. very confident 

2. confident 

3. don’t know 

U. not confident 
f>. very unconfident 
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COURSE EVALUATION* 



Course 



Number NONE DESIRED 



Date 



Name NONE DESIRED 



NOTE TO STUDENTS: Following is a list of factors which are 

important to many courses but over which the instructor often has 
little control. You are to rate the course on each of the factors 
by circling one of the numbers at the right of each statement. 

DO NOT OMIT ITEMS. 




2. Suitability of the size of the class (consider 
the subject matter and type of class— lecture, 



discussion, lab., etc.) * i 2 

3. Amount of freedom allowed students in the 

selection of the materials to be studied 
(considering the subject matter) 1 2 

4. How the course is fulfilling your needs 
(consider your long range as well as your 

short range goals) - 2 

5. The degree to which the objectives of the 

course were clarified and discussed 1 2 



6. The agreement between the announced objectives 
and rules of the course and what was actually 
done — . 

7 • Suitability of the reference materials available 



for the course „ ^ 2 

8. Suitability of the laboratory facilities 

available for the course-------------- ... -- 1 2 

9. Suitability of the assigned textbook -— — — — — 1 2 

10. The use made of tests as aids to learning 1 2 



11. Range of ability in the class (are there too 

many extremely dull or extremely bright 
students) — 

12. Suitability of the amount and type of assigned 

outside work------- 



3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 4 5 

3 k 5 

3 4 5 

3 4 5 



* These sheets and summaries thereof are a matter of PUBLIC RECORD. 



OVER 




13. The weight given to test in determining the' 

final grade for the course------- — -- 1^*2 3 4 

14. Coordination of the tests with the major 

objectives of the course-- * — -1 2 3 4 5 

15. Frequency of tests- — 2 3 4 5 

16. Interest in subject matter—————— — — — 1 2 3 4 5 

17. Suitability of the record requirement 1 2 3 4 5 

18. Suitability of confidence level estimates- — 1 2345 

19 c Suitability of the "CLASS POLICY" paper 1 2 3 4 5 

20. Suitability of the self evaluations — — — 1 2 3 4 5 

21. Suitability of THE COURSE - 1 2 3 4 5 

22. Suitability of the Peer evaluation -1 2 3 4 5 

23. The OVERALL RATING of the course 1 2 3 4 5 

SUMMARY of your ratings: # of l's ; # of 2's ; # of 3's 

# of 4‘s ; # of 5's 

M G ds Mean Median 



What have you liked especially well about this course? 



What have you especially disliked about this course? 



What might be done to improve this course? 



Circle your answer 

24. Amount of time spent on this course 

ABOVE AVERAGE AVERAGE BELOW AVERAGE 

25. Do you think that your instructor should have asked your opinion 
of this course? 

YPg NO NO OPINION 



26 . 



Sex: 



MALE 



FEMALE 



Figure h 

INSTRUCTOR EVALUATION* 

Course Number NONE DESIRED 

Date Name NONE DESIRED 

Listed below are several qualities which describe aspects of the 
instructors behavior. Rate the instructor on each of these items by 
drawing a circle around the number that best indicates his position in 
comparison with other teachers you have had. Rate each item as 
thoughtfully and carefully as possible. DO NOT OMIT ITEMS. 

1. Interprets abstract ideas and theories clearly. a l 234 5 \ 

2. Gets me interested in his subject. 1 2 3 4 5 

3. Has increased my skills in thinking. 1 2 3 4 5 

4. Has helped broaden my interest. — * 1 2 3 4 5 

5. Stresses important material. , 1 2 3 4 5 

* y 

6. Makes good use of examples and illustrations. — 1 2 3 4 5 

7. Has motivated me to do my best work. 1 2 3 4 5 

S<, Inspires class confidence in knowledge of subject. -** 1 2 3 4 5 

9. Has given me new viewpoints or appreciations. 1 2 3 4 5 

10. Is clear and understandable in his explanations. 1 2 3 4 5 

11. His method of teaching. 1 2 3 4 5 

12. Personal appearance. 1 2 3 4 5 

13. Sense of proportion and humor. “ 1 2 3 4 5 

14. Tells you in detail what to learn. ; 1 2 3 4 5 

15. Allows for individual goals. — * 1 2 3 4 5 

16. Fairness in grading. — ‘ " ° 1 2 3 4 5 

17. Sympathetic attitude toward students. 1 2345 

18 o Instructor's opinions.- 1 2 3 4 5 

19. Speech fluency. — - — - — ***’”“' 1 2 3 4 5 

20. The OVERALL RATING of the instructor. 1 2 3 4 5 

SUMMARY : #of l*s ; # of 2’s ; # of 3's ; # of 4's ; 

# of 5's ; Median ; Mode ; Mean ; 



*These sheets and summaries thereof are a matter of PUBLIC RECORD, 



What specific thing did the instructor do especially v?ell in 
his teaching of this course? 



What specific thing did the instructor do worst in his teaching 
of this course: 



What specific things do you beleive might be done to improve 
his teaching of this course? 



CIRCLE YOUR ANSWERS 

> 

21.. Do you want to return to a "Traditional" approach (multiple 
guess, true-false, etc. questions; and lectures): 

YES NO NO OPINION 

22. Would you rehire the instructor? 

YES NO NO OPINION 

23. I am learning things of: 

PRACTICAL VLAUE SOME PRACTICAL VALUE NO PRACTICAL 

VALUE 

24. I expect to receive a grade of: 

A B C D F I W 

25. Would you like to take a course under this instructor again? 

GLADLY NO OPINION WITH HESITATION NEVER 

\ 

26. Do you think your instructor should have asked your opinion 
of his teaching? 

YES NO NO OPINION 



27. MALE FEMALE 



COMMENTS : 
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ALL SUBJECTS— -BY SEMESTER 

DATE COMPLIED 03«20~69 



.STUDENT EVALUATIONS OF JON GOSSER 
BY ALL STUDENTS AS QUALIFIED BELOW 



PART OF SEMESTER'— END 

NAME OF EVALUAT I GN«"COURSE EVALUATION 



DIRECTIONS 



COURSE- 
DATE 






NUMBER NONE DESIRED 

= <cict»2»djc»>aDt so m> *sv «e» m? 

NAME NONE DESIRED 



|B0 






NOTE TO* STUDENTS FOLLOWING IS A LIST OF FACTORS WHICH ARE 
IMPORTANT TO MANY COURSES BUT OVER WHICH THE INSTRUCTOR OFTEN HAS 
LITTLE CONTROL. . YOU ARE TO RATE THE COURSE ON EACH OF THE FACTORS 
BY CIRCLING ONE OF THE NUMBERS AT THE RIGHT OF EACH STATEMENT. 

DO NGT OMI-T ITEMS. 
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CIRCLE YOUR ANSWER 

DG YOU THINK- THAT YOUR INSTRUCTOR SHOULD HAVE ASKED YOUR 
OPINION OF THIS COURSE 





YES 




NO 




NO i 


OPINION 








1=YES 


, 2=N0 r 


3=NC 
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